Entropy Semiring Forward-backward Algorithm for HMM Entropy Computation
نویسنده
چکیده
The paper presents Entropy Semiring Forwardbackward algorithm (ESRFB) and its application for memory efficient computation of the subsequence constrained entropy and state sequence entropy of a Hidden Markov Model (HMM) when an observation sequence is given. ESRFB is based on forwardbackward recursion over the entropy semiring, having the lower memory requirement than the algorithm developed by Mann and MacCallum, with the same time complexity. Furthermore, when it is used with forward pass only, it is applicable for the computation of HMM entropy for a given observation sequence, with the same time and memory complexity as the previously developed algorithm by Hernando et al.
منابع مشابه
Entropy Message Passing Algorithm
Message passing over factor graph can be considered as generalization of many well known algorithms for efficient marginalization of multivariate function. A specific instance of the algorithm is obtained by choosing an appropriate commutative semiring for the range of the function to be marginalized. Some examples are Viterbi algorithm, obtained on max-product semiring and forward-backward alg...
متن کاملIncorporation of HMM output constraints in hybrid NN/HMM systems during training
This paper describes a method to incorporate the HMM output constraints in frame based hybrid NN/HMM systems during training. While usually the NN parameters are adjusted to maximize the cross-entropy between the frame target probabilities and the network predictions assuming statistically independent outputs in time, in the approach described here the full likelihood of the utterance(s) using ...
متن کاملDeterministically annealed design of hidden Markov model speech recognizers
Many conventional speech recognition systems are based on the use of hidden Markov models (HMM) within the context of discriminant-based pattern classification. While the speech recognition objective is a low rate of misclassification, HMM design has been traditionally approached via maximum likelihood (ML) modeling which is, in general, mismatched with the minimum error objective and hence sub...
متن کاملThe Entropy Message Passing: A New Algorithm Over Factor Graphs
Message passing over a factor graph can be considered as a generalization of the many well-known algorithms for an efficient marginalization of multivariate functions. The specific instance of the algorithm is obtained by selection of an appropriate commutative semiring for the range of the function to be marginalized. Some of the examples are the Viterbi algorithm, obtained on a max-product se...
متن کاملIvestigation of Entropy Generation in 3-D Laminar Forced Convection Flow over a Backward Facing Step with Bleeding
A numerical investigation of entropy generation in laminar forced convection of gas flow over a backward facing step in a horizontal duct under bleeding condition is presented. For calculation of entropy generation from the second law of thermodynamics in a forced convection flow, the velocity and temperature distributions are primary needed. For this purpose, the three-dimensional Cartesian co...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1108.0347 شماره
صفحات -
تاریخ انتشار 2011